Rec'i PIT/PT8 1 0 MAY 2005 



(12) 



UK Patent Application ns,GB ,,,,2372 656 ,,3, A 



(43) Date of A Publication 28.08.2002 



(21) Application No 0104456.9 

(22) Date of Filing 23.02.2001 



(71) Applicant(s) ^ 

Industrial Control Systems Limited 
(Incorporated In the United Kingdom) 
4 Cockshutt Drive. Greenhill SHEFFIELD, S8 7DW, 
United Kingdom 

(72) Inventor(s) 

John Michael Petty 
Phillip Stephen Petty 

(74) Agent and/or Address for Service 
Franks & Co 

9 President Buildings, Savile Street East, SHEFFIELD, 
S4 7Ua United Kingdom 



(51) INTCL^ 

G01S 17/46 



10/534494 



(52) UK CL (Edition T ) 

H4D DLRA D23X D234 D265 D710 D733 D745 D747 
0748 D752 D757 D773 D775 D782 D783 



(56) Documents Cited 
EP 0224237 A1 
US 5915033 A1 



EP 0145957 A1 



(58) Field of Search 

UK CL (Edition S ) G1A AAJL , H4D DLAT DLM DLPC 
DLPX DLRA 

INT CL^ G01C 3/08 3/10 , 601S 7/491 17/46 



(54) Abstract Title 

Optical position determination 

(57) An optical rangefinder.projects a number of beams of light onto a target surface in a preset pattern. This 
is viewed by a camera having a CCD or CMOS sensor array, and from the positions of the reflected light on the 
sensor array the orientation of the camera array relative to the target surface is determined. This is used to 
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The colour of the light beams may be used to identify reflected light. 
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APPARATUS AND METHOD FOR OBTAINING THREE-DIMENSIONAL 
POSITIONAL DATA FROM A TWO-DIMENSIONAL CAPTURED IMAGE 

Field of the Invention 

5 The present invention relates to apparatus and methods for determining 

multi-dimensional positional data from a two-dimensional image and particularly, 
although not exclusively, to apparatus and method for detennining three- 
dimensional positional data from a two-dimensional captured image of a field of 
view comprising a plurality of optical markers. 

10 

Background to the Invention 

Prior art cameras incorporate a light-sensitive media in the fonn of an 
insertable film on which the image captured by the camera is formed. As a 
general mle, the larger the film format, the finer the quality of image. Such films 

15 use a chemical process to form the image, the image being fixed onto a hard 
copy format e.g. paper. Such films produce an image as a gradient, the image 
not being formed by a number of identifiable and measurable image quanta. 
From this traditional chemical film format, semi-conductor an^ys have been 
developed for capturing images via the sensing of light and production of a 

20 plurality of digitised signals. Examples of such digital image plates known in the 
prior art are the CCD (Charged Couple Device) and CMOS (Complementary 
MOS) arays. 

CCD and CMOS arrays comprise a plurality of photo-sensitive elements 
25 which each generate a pixel of data. Each pixel of data is then combined with 
adjacent pixels of data to produce an image of the entire field of view. Each pixel 
therefore has measurable physical dimensions which can be determined. These 
physical dimensions can be related to known positions in the field of view. 



Prior art devices and methods have taken advantage of the CCD array to 
accurately measure distances between a digital camera and objects in a field of 
view. This type of range finding is illustrated in the following prior art documents: 

• GB 2344012 

• EP 0205175 

• DE 4238891 

• JP 07294216 

• JP 08005370 

• JP 08094324 

• JP 09304055 

• DE 19736588 

• DE 29921143 U1 

• DE 19949838 

• JP 08285633 

• JP 11094520 

• JP 2000121354 

• US 6,094,270 

The above referenced prior art documents are concerned with measuring 
distances from an object to an image plate or lens including methods of 
eliminating infomiation of low reliability. 

In these prior art documents it is apparent that range finding has become 
possible using CCD image plates and a marker introduced to the field of view. 
The CCD then enables the pixel dimensions to be determined, and determining 
the characteristics of the markers introduced to the image enables trigonometrical 
surveys and Pythagoras calculations to obtain the distance from the Image plate 
to the object in view. 
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EP 0205175 discloses the use of a plurality of cameras to obtain distance 
information relating a robot arm to an object. Detennination of ami position is 
made as a relative determination between known points and does not provide 
Information for all points in the field of view of one camera. Additionally, a 
5 plurality of cameras are required to provide image Infomnatlon in more than two 
dimensions. 

US 6,094,270 is concerned with real time range finding. A sweeping light 
projection is required and an analysis of reflective light intensity is required to be 
10 adjusted to account for changes in reflectance characteristics of a surface 
dependent on the wavelength of incident light. It is necessary for a sweep of light 
to traverse the entire field of view to establish positional data about that field of 
view by a simple range finding to each point swept by the beam of light. 

15 There is therefore a problem with apparatus and methods known in the prior 

art in that the prior art teaches that it is not possible to simply introduce a plurality 
of optical markers into a field of view, that field of view then being captured by an 
image plate, wherein the said optical markers have known optical characteristics, 
to obtain three-dimensional positional data about objects and surfaces (being 

20 points of reflectivity) in the field of view by an analysis of the two-dimensional 
captured image. The present invention solves this technical problem by providing 
technical apparatus and methods for the determination of three-dimensional 
positional data describing the position of regions of reflectivity in a two- 
dimensional captured image of a field of view. 

25 

Summary of the Invention 

The inventors have realized that the projection of a plurality of markers 
having kno\Am characteristics can enable a two-dimensional image to be captured 
of the field of view including the projected markers such that a trigonometrical 
30 sun/ey can be conducted to establish a set of image plate orientation values 



enabling orientation of each pixel in a pixel array with respect to the real world 
regions of reflectivity in the field of view. This provides infonnation about each 
pixel and the area of the real world field of view captured by the image to which 
each pixel is associated (the pixels worth). By applying a process of projective 
geometry, vector analysis and trigonometry to these orientation values three- 
dimensional positional data in respect of regions of reflectivity in the field of view 
can be established. 

According to a first specific method of the present invention there is 
provided a method of obtaining multi-dimensional positional data from a field of 
view, said field of view containing a plurality of regions of reflectivity, said method 
comprising the steps of: projecting into said field of view a plurality of detectable 
markers, each mari<er produced by incidence of an electromagnetic beam on a 
region of reflectivity, wherein an angular relationship between said beams is 
known and a spatial relationship between the plurality of beams is also known at 
a source of said beams; and capturing a two-dimensional image of said field of 
view and said markers on an image plate, said image plate comprising an aray 
of elements each capable of generating a pixel of data; and identifying said 
mariners in the captured image and using said known spatial and angular 
relationships to detemiine a set of orientation values describing the orientation of 
said Image plate to regions of reflectivity in said field of view; and using said 
orientation values to reconstmct a set of multi-dimensional data describing the 
position of said regions of reflectivity. 

Preferably, multi-dimensional comprises uni-dimensional, two-dimensional 
or three-dimensional. 

Preferably, the spatial and angular relationship between said image plate 
and said sources is also known. 
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Preferably, said image plate comprises a CCD or CMOS array. 

Preferably, said orientation values comprise the pan of said image plate 
with respect to regions of reflectivity in said field of view, the pan being the 
.5 deviation of the image plate from an opposite parallel plane. 

Preferably, said orientation values comprise the twist of said image plate 
with respect to regions of reflectivity in said field of view, the twist being the 
deviation of the image plate from an adjacent parallel plane. 

10 

Preferably, said orientation values comprise the tilt of said image plate with 
respect to regions of reflectivity In said fleld of view, the tilt being the deviation of 
the image plate from a plane at right angles to said Image plate. 

15 Preferably, said orientation values comprise at least one of the pan, twist, 

tilt and height of said image plate with respect to regions of reflectivity In said field 
of view. 

Preferably, wherein said step of using said orientation values to 
20 reconstruct a set of multi-dimensional data describing the position of said regions 
of reflectivity comprises the steps of: using said orientation values to evaluate 
each pixel In the real world by detemilning the degree of pan and/or twist and/or 
tilt for each pixel; associating each said pixel of data with a said regbn of 
reflectivity and detemiining the real worid position of said region of reflectivity. 

25 

Preferably, the method further comprises the step of: repeating said 
evaluation of each pixel for heights above a zero plane. 

Preferably, the method further comprises the step of selecting a pixel 
30 having known three-dimensional positional data and scanning to all three- 



dimensional planes to test that each pixel nnatches a set of three-dimensional co- 
ordinates. 

Preferably, said step of using said orientation values to reconstruct a set of 
three-dimensional data describing the three-dimensional position of said regions 
of reflectivity further comprises the steps of: applying an edge detection function 
to the two-dimensional captured image; and checking the three-dimensional field 
data against the edge detection results to establish a finalised set of three- 
dimensional positional data describing the three-dimensional position of said 
regions of reflectivity. 

Preferably, said step of identifying said markers in the captured image 
comprises the step of: conducting a search of the captured two-dimensional 
image to identify pixels receiving reflected light of a particular wavelength. 

Preferably, said method further comprises the step of mapping the three- 
dimensional data obtained to reconstruct a virtual three-dimensional 
representation of the field of view. 

Preferably, said optically detectable marker captured in said two- 
dimensional image is configured to cover an area of between one and four pixels 
on said image plate. 

Preferably, said orientation values comprise one or more vector 
measurements of said regions of reflectivity. 

Preferably there is provideda computer program product directly loadable 
into the internal memory of a digital computer comprising software code portions 
for performing the steps of: identifying said markers in the captured image and 
using said known spatial and angular relationships to calculate a set of 
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orientation values describing the orientation of said image plate to regions of 
reflectivity in said field of view; and using said orientation values to reconstruct a 
set of multi-dimensional data describing the position of said regions of reflectivity; 
said steps according to the first specific method described herein. 

5 

Preferably, a computer program product directly loadable into the internal 
memory of a digital computer comprising software code portions for perfomning 
any of the steps according to the methods of the present invention. 

10 According to a second specific method of the present invention there is 

provided a method of determining the distance between regions of reflectivity in a 
field of view and an image plate, said method comprising the steps of: projecting 
into said field of view at least one detectable mariner, said marker produced: by 
incidence of an electromagnetic beam on a region of reflectivity; wherein angular 

15 and spatial relationships between the source of said beam and said image plate 
are known; and capturing a two-dimensional image of said field of view and said 
markers on an image plate, said image plate comprising an array of elements 
each capable of generating a pixel of data; and identifying said marker in the 
captured image and using said known spatial and angular relationships to 

2 0 determine a set of orientation values describing the orientation of said image 
plate to regions of reflectivity in said field of view; and using said orientation 
values to determine the distance between said image plate and one or more of 
said regions of reflectivity. 

25 According to a first aspect of the present invention there is provided 

apparatus for the projection of a plurality of detectable markers onto a field of 
view, wherein said markers are capturable as part of a two-dimensional image of 
said field of view fonned on an image plate, said image plate comprising an aray 
of elements each capable of generating a pixel of data, said apparatus 

30 comprising: a projection unit comprising at least one electromagnetic radiation 



source, said at least one source configured to produce a plurality of 
electromagnetic beams, each beam producing a said detectable marker on 
incidence with a region of reflectivity in said field of view, wherein an angular 
relationship between the projected beams Is known, a spatial relationship 
between the plurality of beams is also known at a source of said beams; and 
mounting means configured to engage said projection unit; and said mounting 
means further comprising means to locate said image plate in a fixed spatial and 
angular orientation; wherein said projected beams are configured to project said 
markers onto said field of view to provide Information for the analysis of a 
captured two-dimensional Image of said field of view and markers in order to 
establish a set of orientation values describing the orientation of said image plate 
to regions of reflectivity in said field of view, said orientation values enabling the 
reconstruction of multi-dimensional data describing the position of regions of 
reflectivity in said field of view. 

Preferably, multi-dimensional comprises uni-dimensional, two-dimensional 
or three-dimensional. 

Preferably, said image plate comprises a CCD or CMOS an^y. 

Preferably, said image plate is comprised by a digital camera or scanner. 

Preferably, said plurality of beams comprises at least four beams. 

Preferably, said markers fomri a predefined pattem on said field of view. 

Preferably, said electromagnetic radiation comprises radiation of a 
wavelength in the range 10'^^ m to lO^* m. 
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Preferably, said electromagnetic radiation comprises visible light of a 
wavelength of 635nm. 

Preferably, said image plate is comprised by a digital camera, said 
5 mounting means configured to engage said digital camera for location of said 
digital camera in fixed spatial and angular orientation to said projection unit. 

According to a second aspect of the present invention there is provided a 
device for projecting a plurality of markers onto a field of view, said markers 

10 capturable on an image plate comprising an array of elements each capable of 
generating a pixel of data, the captured two-dimensional image configured for the 
reconstruction of a set of multi-dimensional data describing the position of regions 
of reflectivity in said field of view from said captured two-dimensional image, said 
device comprising: at least one electromagnetic radiation source, said source 

15 configured to produce at least one electromagnetic radiartion beam, said beam 
producing a said marker on incidence with a region of reflectivity in said field of 
view; and means to focus said beams; wherein an angular relationship between 
said beams and a spatial relationship between said sources is known, the 
markers thereby forming a predefined pattern In said field of view. 

20 

Preferably, said electromagnetic radiation source comprises at least one 

laser. 

Preferably, said electromagnetic radiation source produces light of a 
2 5 wavelength In the range 1 0'^^ m to 1 0"^ m. 

Preferably, said electromagnetic radiation source produces visible light of 
a wavelength of 635nm. 



3 0 Preferably, said Image plate fornis part of a digital camera. 
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Preferably there is provided a digital camera comprising a device 
according to the second aspect of the present invention wherein said image plate 
is fornied by the image plate housed within said digital camera. 

According to a third specific method of the present invention there is 
provided a method of determining a set of calibration values for an image plate 
comprising an array of elements each capable of generating a pixel of data, said 
image plate configured for use in capturing a two-dimensional image of a field of 
view comprising a plurality of markers, said set of calibration values enabling the 
detemilnation of multi-dimensional positional data describing the position of 
regions of reflectivity in said field of view, said method comprising the steps of: 
locating an imaging device in a first fixing position, said imaging device housing 
said image plate; and capturing a two-dimensional Image of a reference plane; 
wherein said image plate is substantially parallel to said reference plane. 

Preferably, following capture of an image at a first fixing position, said 
imaging device is relocated in at least one second fixing position from which at 
least one second image is captured. 

Preferably, said reference plane comprises a planar surface divided into a 
grid having known dimensions. 

Preferably, said image plate comprises a CCD or CMOS array. 

Preferably, said method further comprises the step of establishing a set of 
calibration values specific to said image plate. 

Preferably, said set of calibration values comprise one or more values 
taken from the set of: X. Y and Z offsets of said Image plate from the fixing 



position; x, y pixel dimensions; focal length; image plate width and height; x and y 
image plate view angles. 

According to a third aspect of the present Invention there is provided 
apparatus for determining a set of calibration values for an image plate 
comprising an array of elements each capable of generating a pixel of data, said 
image plate configured for use in capturing a two-dimensional image of a field of 
view comprising a plurality of mariners, said set of calibration values enabling the 
detemnination of multi-dimensional positional data describing the position of 
regions of reflectivity in said field of view, said apparatus comprising: means \g. 
locate an imaging device in at least one fixing position, said imaging device 
housing said Image plate; and a reference plane located in said field of view. 

Preferably, said reference plane comprises a planar surface substantially 
parallel to said image plate and further comprising a grid of known dimensions. 

Preferably, said image plate comprises a CCD or CMOS array. 

According to a fourth specific method of the present invention there Is 
provided a method of using a single camera device to obtain three-dimensional 
positional data from a two-dimensional captured image, said two-dimensional 
captured image including a plurality of non-intmsive optical markers having 
known characteristics, said optical mariners being capturable in a two-dimensional 
image of the field of view. 

Brief Description of the Drawings 

For a better understanding of the invention and to show how the same may 
be carried into effect, there will now be described by way of example only, 
specific embodiments, methods and processes according to the present 
invention with reference to the accompanying drawings in which: 
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Fig. 1 illustrates a diagrammatic representation of the apparatus used to 
capture a two-dimensional image and to analyse the captured image to obtain 
threeHdimensional positional data. 

5 

Fig. 2A illustrates a first predefined pattern of optical markers. 

Fig. 2B illustrates a second alternative predefined pattern of optical 
markers. 

10 

Fig. 3A illustrates in side view a diagrammatic representation of the 
projection of optical markers towards a plane. 

Fig. 3B illustrates in diagrammatic representation a projection of beams to 
15 produce a plurality of optical markers at a plane. 

Fig. 3C illustrates a pattern of optical markers produced at a plane. 

Fig. 4 illustrates a diagrammatic representation of an image plate and 
20 projection unit projecting two light beams towards a surface for the production of 
optical markers and further illustrates reflection and image capture of the optical 
markers produced. 

Fig. 5 illustrates a diagrammatic representation of an active image plate. 

25 

Fig. 6A and Fig. 6B illustrate two-dimensional representations of a virtual 
cone deriving the x and y pixel angle fomied on the central axis between the focal 
point and a circle encompassing the image plate of an imaging device. 
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Fig. 7 illustrates a diagrammatic representation of an image plate and focal 

point. 

Fig. 8 illustrates a diagrammatic representation of tlie view angle of the 
5 imaging device when tilted. 

Fig. 9 is a diagrammatic representation of projected light beams towards a 
surface for the determination of distance between optical markere. 

10 Fig. 10 illustrates an internal plan view and end elevation of a first 

embodiment of a projection unit. 

Fig. 1 1 illustrates a plan view, end elevation and front elevation of a casing 
for a projection unit. 

15 

Fig. 12 illustrates a plan view and end elevation of a combined projection 
unit and camera apparatus. 

Fig. 13A Illustrates a side elevation of a mounting means. 

20 

Fig. 13B illustrates a front elevation of a mounting means. 

Fig. 14 illustrates table 1 comprising a sample laser specification. 

25 Fig. 15 illustrates a flow chart of the basic steps involved in obtaining three- 

dimensional positional field data. 

Fig. 1 6 illustrates table 2 listing sample calibration infomiation. 
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Fig. 17 illustrates a diagrammatic representation of the calibration apparatus 
and method. 

Fig. 18 illustrates a first calibration flow chart for the determination of a set 
of calibration values. 

Fig. 19 illustrates a second calibration flow chart for the determination of a 
set of calibration values. 

Fig. 20A is a diagrammatic representation of twist. 

Fig. 20B is a diagrammatic representation of tilt. 

Fig. 20C is a diagrammatic representation of pan. 

Fig. 21 illustrates a diagrammatic Illustration to show the change in surface 
area that a pixel in an image plate represents with varying image plate 
orientation. 

Fig. 22 Illustrates a diagrammatic representation of the effect of light beams 
striking a ground plane to produce optical markers having pan and tilt and a 
reflection back to the image plate. 

Fig. 23A, B and C indicate the angular and spatial measurements 
detemiined through vector analysis. 

Fig. 24 illustrates a first flow diagram for the determination of a set of 
orientation values. 
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Fig. 25 illustrates a second flow diagram for the determination of a set of 
orientation values. 

Fig. 26 illustrates a third flow diagram for the determination of a set of 
orientation values. 

Fig. 27 illustrates a fourth flow diagram for the detemriination of a set of 
orientation values. 

Fig. 28 illustrates In diagrammatic representation the effect of varying x, y 
and z on the representative object area associated with a single pixel. 

Fig. 29 illustrates a flow diagram to determine the location of optical markers 
in a captured two-dimensional image. 

Fig. 30 illustrates a first image analysis flow diagram. 

Fig. 31 illustrates a second image analysis flow diagram. 

Fig. 32 illustrates a third image analysis flow diagram. 

Fig. 33 illustrates a fourth image analysis flow diagram. 

Fig. 34 illustrates a flow diagram showing a summary schenne for the 
determination of a set of three-dimensional positional data. 

Fig. 35 is an example of a screen showing calibration data. 
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Flg. 36 illustrates in diagrammatic representation the combination of 
captured images to cover a field of view larger than that in a single captured 
image. 

5 Fig. 37 illustrates the combination of imaging devices to capture an 

extended field of view. 

Detailed Description of the Best Mode for Carrying Out the invention 
There will now be described by way of example the best mode 
10 contemplated by the Inventors for carrying out the invention. In the following 
description numerous specific details are set forth in order to provide a thorough 
understanding of the present invention. It will be apparent however, to one 
skilled in the art, that the present invention may be practiced without limitation to 
these specific details. In other instances, well known methods and structures 
15 have not been described in detail so as not to unnecessarily obscure the present 
invention. 

The inventors have realized that two dimensional photographic images 
captured on an image plate, said image plate comprising an an^y of 

20 photosensitive elements, each element capable of generating a pixel of data, 
when combined with at least one mari<er Introduced to the field of view, enables 
spatial and positional data of objects and surfaces in the field of view to be 
determined. In order to obtain this spatial and positional data it is necessary to 
obtain a representation of the image of the field of view on a quantized image 

25 plate comprising an an-ay of said elements sensitive to the wavelength of the 
mariner. This is necessary because the determination of spatial and positional 
data from said image is resultant on knowing information about the dimensions 
and optical characteristics of each pixel. 




-17- 



The inventors have realized that the introduction of a plurality of optical 
markers into a field of view, each marker produced by a beam of light or 
electromagnetic radiation incident on objects or surfaces being regions of 
reflectivity in said field of view coupled with knowledge of the spatial and angular 
5 relationships between said beams and the source of each beam, and further 
coupled with infomnation relating the image plate on which an image of the field of 
view and said markers is captured, to the source of each beam, and applying a 
trigonometrical sun/ey to establish a set of orientation values, enables image 
analysis of the captured two-dimensional image to produce a set of three- 
10 dimensional positional field data describing the three-dimensional position of 
each region of reflectivity in the field of view. 

The present invention comprises apparatus for the projection of one or a 
plui^llty of optical markers required to obtain Information about the field of view, ; 
15 apparatus for, and a method of, calibrating the image plate on which a two- 
dimensional image of the field view and projected markers is captured, and a 
method of detemiining three-dimensional positional data describing the three- 
dimens'ional position of regions of reflectivity in a field of view of the image plate. 

20 The determination of three-dimensional posrtional data necessarily requires 

the determination of single dimension and two-dimensional data and the methods 
and apparatus of the present invention enable all such data to be determined. 

Refening to Fig. 1 there is illustrated a field of view 101 being the area of a 
25 scene or landscape viewed and capturable by an image plate housed in an 
optical/imaging device 102, typically a digital camera. The field of view for 
instance could be a countryside landscape, a building site, products or parts and 
machinery fomning part of a production line, people in a busy high street and 
generally any objects which can be viewed through an imaging device and 
30 captured on an image plate. Fig. 1 illustrates the apparatus required to obtain 
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information required to calculate three-dimensional positional data of regions of 
reflectivity within the field of view 101. A mounting means 104 is provided upon 
which imaging device 102 is engaged in fixed position. One method of engaging 
imaging device 102 upon the mounting means 104 is to make use of the tripod 
mounting on a digital camera. Mounting means 104 also provides rneans to 
locate a projection unit 103. Mounting means 104 is required to locate projection 
unit 103 and imaging device 102 in fixed, spatial and angular orientation to each 
other. Mounting means 104 may further be mounted on a tripod or provide for 
hand held operation of the combined imaging device, mounting means and 
projection unit 103. 

Once an image has been captured, e.g. by taking a photograph, on the 
image plate of imaging device 102, said image comprising the field of view 101 
and optical markers 107. this Image is downloaded to a PC or laptop 108 where 
the two-dimensional captured image can be displayed 109 and analysed. 

Projection unit 103 comprises at least one light source 105. Said at least 
one light source is configured to produce a plurality of beams of light 106. One or 
more beam splitters may be used. Upon incidence of each beam of light on a 
region of reflectivity in the field of view an optical marker 107. preferably a light 
spot is produced. Each light source is at a fixed spatial position which when 
projection 103 is fixedly mounted on mounting means 104. provides for each 
source to be in a fixed spatial position relative to imaging device 102 which is also 
mounted on the mounting means 104. The beams of light 106 are produced at 
fixed angular relationships to produce a pre-defined pattern of optical mariners in 
the field of view. 

The source of ttie light beam is required to be known such that fixed angular 
and spatial relationships can be calculated. Provided that the position of the light 
source Itself and any lens means provided to focus the light beam are known, the 
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source can be considered as either the point at which the laser originates or the 
holographic or coliimated or other lens used to focus the light beann. That is 
provided a plane of origin of the light/electromagnetic radiation beam can be 
determined, then this position can be considered the source of the beam having 
5 known fixed spatial and angular relationships to any other sources and to the 
Image plate thereby enabling calculation of a set of orientation values and image 
analysis. 

An example of 2 predefined patterns of optical mariners is shown in Fig. 2 A 
10 and B. Figure 2A illustrates 6 optical mariners, mariners A and D, C and F 
approximately forming the comers of a rectangle and markers B and E being 
inwardly offset to the corners of the said rectangle towards A, D wherein B, E can 
potentially cross or overiap A, D. Fig. 2B illustrates an alternative an-angement of 
optical markers there being 5 mariners anranged in a + shape. The invention is 
15 not limited by the shape fonned by the optical markers but the fixed angular 
relationships between the beams producing each mariner and the sources 
producing each beam is required to be known. 

To obtain a full set of orientation values comprising height, pan, tilt, twist it is 
20 necessary to have a minimum of 4 optical markers. However, projecting a pair of 
mariners provides sufficient information to calculate the tilt or pan, as well as the 
height and is thereby sufficient to give detailed and accurate two-dimensional 
positional data of ail of the regions of reflectivity in the field of view. Projecting 
only a single optical marker provides sufficient infonnation to obtain the 
25 orientation value of height of the image plate with respect to each region of 
reflectivity and thereby provides a distance or range finding function for each 
region of reflectivity in the field of view. The obtaining of a three-dimensional data 
set necessarily includes the determination of height and two-dimensional data, 
the method of detennining a three-dimensional data set is described. 

30 
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To obtain sufficient data to reconstruct a three-dimensional data set 
describing the position of regions of reflectivity in a field of view, a minimum of 4 
optical markers are required. Each marker is produced from a single light beam 
incident on a region of reflectivity. A single light source may be split using a 
graticule or crystal into the requisite number of light beams. 

It Is possible to create a large single optical marker within the field of view. 
The resultant distortion of a single large optical marker can in effect be treated as 
though a plurality of markers are present formed by the differing extent of 
distortion at the edges of the marker. The use of such a single optical marker is 
in effect fonning a plurality of optical markers which provide infonnation from 
which a set of orientation values describing the orientation of the image plate to 
the regions of reflectivity in the field of view can be established. 

Considering the rectangular configuration of markers illustrated in Fig. 2A 
projected into a field of view. Distortion of the projected rectangle into the field of 
view is a function of the effects of pan, tilt and twist of the image plate with 
respect to the regions of reflectivity in the field of view. These orientation values 
enable a set of orientation values describing the orientation of the image plate 
with respect to the regions of reflectivity in the field of view to be determined. 
Image analysis can then be conducted to establish multi-dimensional e.g. two 
dimensional or three dimensional positional field data. 

Fig. 3B illustrates the path taken by each beam 106 to produce each optical 
marker 107 in the field of view 101. Fig. 3A shows a side on view of projection 
unit 103 projecting beams 106 onto a ground surface or plane 101. Fig. 3 B 
illustrates the projection of 6 beams 106 producing 6 optical markers A, B, C, D. 
E, F (107). Fig. 3 C illustrates the optical markers 107 as projected onto a field of 
view 101 (refer to Fig. 2A). 
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Fig. 4 illustrates a combined imaging and projection unit 401 housing an 
image plate 402 and projection unit 103. Fig. 4 provides an illustration of a 
schematic representation of the relationship between the image plate and 
projection unit. Light beams 106 are projected from light sources within 
projection unit 103 towards surfaces in field of view 101 of image plate 402. 
Regions of reflectivity in the field of view 101 reflect beams 106, reflective beams 
being captured on image plate 402. Regions of reflectivity within the field of view 
101 are not required to be planar and may have dimensions in all 3 x, y and z 
axes. 

In order to obtain three-dimensional positional data which describes regions 
of reflectivity in a field of view, each region of reflectivity being a point in the field 
of view which can be described in three dimensions, it is necessary to capture an 
image of that field of view with appropriate mariners on a quantized image plate. 
Such a quantized image plate may comprise an Image plate having an an-ay of 
photo-sensitive elements, each element capable of generating a pixel of data. 
Imaging devices such as digital cameras and scanner's and digital video 
camera's are known which can acquire an image and convert that image to a 
plurality of digital signals. To do this It is known to form an Image plate from a 
CCD (charged couple device) or CMOS (complimentary MOS) semi-conductor 
array comprising a plurality of photo-sensitive elements. In such an array each 
photosensitive element can produce a pixel of data by converting reflected light 
from a field of view into electrical energy. A CCD for instance converts incidental 
electromagnetic radiation e.g. light, which falls on the surface of the CCD into 
electronic signals which are then stored in a digital fomn in the camera's memory. 
Various types of array are available, one surface an-angement of a CCD is 
configured in the forni of an array 1cm by 1cm (Bayer matrix) forming a grid of 
pixel's. The Image plate array can be of any type provided it is an array 
comprised of materials sensitive to electromagnetic radiation and aranged as an 
an-ay of individual cells. Importantly, the CCD Is a photo-responsive media which 
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has measurable physical dimensions, including x, y and z physical dimensions. 
An image plate comprised by an an-ay of pixel's therefore fomis a two- 
dimensional image of a field of view which can be divided up into individual pixel's 
having physical dimensions which can be related to that field of view. 

An imaging device such as a digital camera therefore has a plurality of 
optical characteristics including focal length, image plate dimensions and pixel 
dimensions. Fig. 5 illustrates a representation of an image plate 501 having an 
image plate height (ly), an image plate width (Ix) and image plate diagonal (Id). 
For a 35 mm photo type format the relevant dimensions will be as follows: 

• Ix = 36 mm 

• ly = 24 mm 

• Id = 43.266615 mm . 
and 

• Id = (Ix^ + Iy2) Equation 1 . 

Between the image plate and lens of a camera there is produced a virtual 
cone with a height equal to the focal length of the camera. Fig. 6A and Fig. 6B 
illustrate, as planar line drawings, representations of this cone wherein F = the 
focal point, the vertical distance between the image plate center and the focal 
point giving the focal length, f, 601 . 

Another representation of the relationship between view angle and image 
plate is shown in Fig. 7. The focal point F is located at the lens. Distance 601 
between F and the center of the image plate being the focal distance, f, and the 
angle (p is the view angle. A relationship of the focal length to the angular field of 
view is given by the angle subtended at the lens with respect to a circle 
encompassing the image plate 502. 



-23- 

Fig. 8 illustrates a diagram showing the limit of the field of view which is 
determined by the angle of view cp. The limit of the field of view 806 is bisected 
by the center axis 805 of the image plate 803. A tilt angle 802 is defined between 
the center axis 805 and the strike plane 801. The field of view includes the 
5 ground plane 801 and the distance between the center of the image plate and 
the ground plane 801 is given by the height, h. If the physical dimensions of each 
pixel in the image plate are measured and the angular orientation of the Image 
plate is known, each pixel can then be transposed to a reference plane, the 
ground plane, and each pixel row and the y plane of the image plate will then 
10 correspond to the position of the y co-ordinate of the image in the real world. In 
this way it Is possible to detemnine the co-ordinates of optical markers 107 which 
may be introduced to the field of view. 

Fig. 9 further illustrates this principal. Fig. 9 illustrates three planes, the 

15 Image plane (I), a plane at which light is projected from a projection unit 103 (L) 
and an object plane being a plane in the field of view (O). Beams of light are 
produced at projection unit 103 projecting beams towards plane O to produce a 
plurality of optical mari<ers 107. Knowing the spatial off-set of the projection unit 
light sources from the image plate I and knowing the angular relationship of 

20 beams projected from the projection unit a set of calibration values can be 
detemiined such that the angular relationship of the markers from the image plate 
can be calculated. In this way a nomialization process can be carried out to 
allow orientation and imaging processes to consider each beam as though it had 
been projected from the image plate such that trigonometrical sun/eys can be 

25 conducted to find out spatial and positional data of regions of refiecfivity in the 
object plane (O). Once the calibration process has been earned out (calibration 
process described below) and where infomnation regarding pixel dimensions, 
image plate dimensions and the spatial and angular offsets of the light sources 
and projection unit 103 from the image plate are known such that distances a and 

30 b are known it is a matter of simple trigonometry to establish the distances (a + A) 
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and B. This information can be obtained due to the fact that each pixel has a 
measurable physical dimension which can be related to the position of markers in 
a field of view. 

5 Fig. 9 is a simplistic representation to Illustrate that distances between 

optical markers in a field of view e.g. distance B can be obtained. Typically, an 
image plate will not capture an image of a field of view that is planar or parallel to 
the image plate. As a result it is necessary to calculate a set of orientation values 
which describe the angles of approach of the Image plate with respect to each 

10 region of reflectivity in relation to each of the three-dimensional axes. This 
allows for the image area occupied by each individual pixel to be analyzed by a 
trigonometrical survey of the relationship between the orientation values 
computed and the optical mariners 107 introduced to the field of view which have 
known angular and relative spatial relationships. By performing a geometrical 

15 vector analysis, x, y and z values can be assigned to each region of reflectivity 
within the field of view. 

Fig. 10 Illustrates in plan view and in end elevation an internal an^ingement 
of the projection unit 103. The projection unit 103 comprises a body 1001 

20 housing at least one light source 1002, an aperture and lens means 1003 for 
focusing light produced by each light source 1002 lens means 1003 comprises a 
holographic lens 1005 and collimated lens 1004. Light source 1002 is preferably 
a laser e.g. diode laser providing a coherent light source. The purpose of the 
projection unit is to provide an array of light spots (the optical mariners 107) into 

25 the field of view which can be captured by an associated imaging device. The 
relationships between the optical markers produced in the field of view are known 
as they relate to the source 1002 of each beam 106 in the projection unit. By 
placing the projection unit 103 in a mounting means 104 the spatial and angular 
relationship to the imaging device and the distance between the image plate and 

30 the projection unit 103 can be determined. The relationship between projection 
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unit and image plate can then be interpreted by the use of projective geometry, 
trigonometry and vector analysis to calculate a set of orientation values which 
describe the tilt, pan, twist and height of the image plate in relation to the plane 
on which a particular optical marker strikes a region of reflectivity in the field of 
5 view. 

Fig. 1 1 shows an illustration of a plan view of a moulded case, showing an 
intemal an^angement, for a projection unit illustrating a body 1001 and apertures 
and lens means 1003 for transmission of light beams. A front elevation of the 
10 case Is also shown illustrating 3 apertures 1101 corespondlng to aperture and 
lens means 1003 through which light beams can be emitted. Apertures 1102 
provide for fastening means inserted to secure the projection unit together. 

Lens and aperture means 1003 typically comprise a holographic 1005 and 
15 collimated 1 004 lens for focusing of the light beam. The collimated lens has fixed 
optical characteristics which can be used to detemnine the divergence of the 
projected beam. Knowing the angular view of the associated imaging device and 
the divergence of light beams emitted from the projection unit it is possible to 
configure the optical mariners 107 produced in the field of view to con-espond to a 
20 required number of pixel's in the image plate. For instance it is preferable that 
one optical mariner 107 is detected by only one pixel on the image plate. 

The number of pixels that are covered by a single mariner are dependent on 
a number of factors, eg the resolution of the image being taken, the distance of 
25 the image plate to the region of reflectivity (the marker), the height of the 
projection unit from the ground surface. 

Fig. 12 illustrates a second embodiment of the projection unit 103. In this 
second embodiment the projection unit and Imaging device are incorporated in a 
30 single unit. For instance, a digital camera may comprise an image plate fomied 
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by a CCD array and at least one light source producing a plurality of light beams 
for projection into the field of view. 

The combined imaging device and projection unit illustrated in Fig. 12 
comprises a body 1201, at least one aperture and lens means 1204 for the 
transmission and projection of light beams produced by light source 1202, an 
image plate 1203 and camera lens arrangement 1205. Preferably the light 
source 1202 is a laser and the lens arrangement 1204 is a collimated holographic 
lens. Image plate 1203 is preferably a CCD or CMOS array. 

Fig. 14 shows table 1 , being a typical laser specification for use as the light 
source in either the first or second embodiment of the projection unit. 

Fig. 13 A and Fig. 13 B Illustrate one embodiment of the mounting means 
104. In this mounting means 104 embodiment, a chassis 1301 is configured to 
receive a projection unit of the type illustrated in Figs. 10 and 11 in a recess 
1302. Projection unit 103 is thereby securely engaged in recess 1302 to provide 
the projection unit in a fixed position. Electrical connection means 1303 are 
provided to connect a battery or mains power source 1304 to the projection unit 
located in recess 1302. A fixing screw 1305 is located on the chassis 1301 for 
location of an imaging device e.g. a digital camera in fixed spatial and angular 
orientation to the recess 1302 and thereby to a projection unit housed therein. A 
beveled Itnob 1306 is provided to engage fixing screw 1305 with the imaging 
means. Whilst mounting means 104 provides for a combination of a projection 
unit and imaging unit in a hand held manner a tripod fixing point 1307 is also 
provided if the user wishes to locate the apparatus in a fixed position. 

The present invention comprises a method of obtaining three-dimensional 
positional data from a field of view. This method comprises the steps of capturing 
an image of a field of view, contained within the field of view are a plurality of 
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optically detectable markers projected by a projection unit as previously 
described. A two-dimensional image is captured on an image plate having an 
an-ay of pixels. Knowing a set of calibration values a set of orientation values are 
detennined which describe the orientation of the image plate with respect to the 
5 field of view. The orientation of each region of reflectivity in the field of view can 
then be detemiined and the image analysed by trigonometrical surveys, 
projective geometry and vector analysis to produce a three-dimensional data set 
describing the three-dimensional position of each region of reflectivity in the field 
of view, 

10 

The process of detemiining three-dimensional positional field data of 
regions of reflectivity in a field of view can be broken down into four basic steps 
as illustrated in Fig. 15. From a starting position 1501 a two-dimensional image 
has been captured which includes a plurality of optical markers 107. In order to 

15 calculate three-dimensional positional data of regions of reflectivity in the field of 
view it is necessary to know details of the image plate e.g. digital camera. 
Therefore a check for image plate details is made 1502. If no image plate details 
are held it is necessary to perform an image plate calibration procedure 1503 
(see below). Once each calibration procedure has been completed image plate 

2 0 details are known and it is possible to progress to step 2. 

Fig. 15 illustrates step 2 as a simplistic stepwise representation of a 
computer program for running step 2. Initially, a file is opened 1504 comprising 
relevant calibration values 1505 for the image plate. A search for the optical 
25 markers 1 506 is conducted and once identified these calibration points are stored 
1507. 

Having identified calibration points it is possible to move onto step 3 
wherein orientation values of the image plate with reference to the optical 
30 mariners in the field of view 1508 are obtained, these orientation values then 
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being stored as data 1509. The orientation of the innage plate to the plane of 
reflectivity captured by each pixel can then be evaluated and the orientation of 
the Image plate with respect to the many planes that may exist in the field of view 
is detemiined to provide a set of orientation values, 

5 

Having detemnined a set of orientation values it is possible to move onto 
step 4 wherein the three-dimensional positions of the regions of reflectivity 
detected In the complete Image array can be determined 1510 and this data 
stored 1511. This completes the image analysis to produce a three-dimensional 
10 data set describing three-dimensional positional data of regions of reflectivity. 

As described for step 1 in Fig. 15 above it is necessary to determine a set of 
calibration values. Sample calibration values comprise the following: 

15 • X, y and z offsets of each light source ftom the Image plate, these 

offsets being the distance in each of the three-dimensional axes 
from the light source to the Image plate; 

• X and y pixel dimensions; 

• angle of deviation of the Image plate from the normal plane; 
20 • Image plate width and height; 

• x and y image plate view angles; 

• Imaging device focal length. 

This information is readily available from the manufacturer or handbook 
25 Information supplied with most digital cameras. If the Information is not available 
then it is necessary to cany out a calibration method to establish all of the details 
about the imaging device. These details can be stored In a data file. In this way, 
a software package may be provided having calibration values stored for a 
plurality of Imaging devices e.g. several makes of digital camera. The user then 
30 simply selects the digital camera model which has been used to obtain the two- 
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dimensional image and the correct calibration values for a standard mounting 
means and projection unit are retrieved for use in the determination of orientation 
values and three-dimensional positional data of regions of reflectivity in the field 
of view (see Fig. 35). 

Fig. 16 shov\/s table 2 which illustrates a sample selection of commercial 
information available on purchasing a particular make of digital camera. 

In order to obtain a set of calibration values a calibration method and 
calibration apparatus are required. Calibration apparatus comprises a jig of three 
upright fixing posts to which the imaging device, e.g. digital, camera is fixed and a 
planar panel fomnlng a reference plane. A grid of 1cm squares is provided on the 
reference plane. The grid is generated by a set of lasers or alternatively is 
imprinted onto the panel fomiing the reference plane. The jig and reference 
plane are configured such that the image plate and reference plane are 
substantially parallel. 

The method of calibration entails the fixing of the iniaging device to the jig in 
a first position and capturing a two-dimensional Image of the field of view 
comprising the reference plane. 

Refenring to Fig. 17 there is illustrated in schematic form the three fixing 
points 1701, 1702, 1703 on the jig. Image plate 1704 is offset 1705 to the center 
of each fixing point. The distance 1706 between fixing point 1703 and reference 
plane 1707 is measured for each fixing point. At each fixing point a plurality of 
images are captured e.g. ten images per fixing point. 

The infomiation then required to obtain the calibration values is specifically 
limited to design information of typical digital imaging devices, e.g. digital 
cameras and digital video cameras, as well as measurable offset distance 1706 
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of the imaging device to the reference plane. The imaging device data required 
for the calibration process are: 

• Focal length of the camera; 

• Camera photographic resolution (CCD plate size). 

The photographic resolution is checked each time a photograph is analyzed 
and compared back against the camera model being used. 

Figs. 18 and 19 show flow charts for the process of determining a set of 
calibration values. The processes illustrated are preferably implemented by a 
suitable software package. Each of the plurality of Images taken at each fixing 
points 1701, 1702, 1703 is analysed 1801. A iscale calculation Is made relating 
the Image to the fixing position relevant to the focal length 1802. A set of 
measurements are then taken from the image at predefined pixel positions 1803. 
From these measurements and the scale calculation a set of calibration values 
1804 are determined. Sample calibration values comprising the following: 

• The linear distance between the fixing tripod screw on the fixing 
point center and the image plate; 

• X and y pixel dimensions; 

• The offset distance of the center of the image plate to the height of 
the fixing point; 

• The X displacement of the fixing point to the image plate; 

• The photographic view angles; 

• The full size of the active image plate array (image plate width and 
image plate height). 
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The next captured image is then loaded 1805 and the process repeated, 
data from the repeated calibration process being averaged 1806 to produce a set 
of calibration values. 

5 Considering the user the method of calibration is simple. Initially, the 

imaging device, e.g. digital camera, is fixed to fixing point 1, 1701. the image 
plate is aligned parallel to the reference plane 1707. The photograph is taken 
(the image captured). This process is repeated a required number of times. The 
camera is then moved to fixing point 2, 1702, and the process is repeated. The 

1 0 camera is further moved to fixing point 3 and the process repeated again. Having 
captured all of the images from each fixing point the image files are copied to a 
PC and calibration software implementing Fig. 18 is run by the user and the 
average data stored as a set of calibration values. This process is illustrated in 
Fig. 19. The camera photographic resolution, R and size of an-ay are loaded, 

15 1901 . Camera resolution is then set between R = 1 to n where n is the maximum 
resolution, 1902. This camera resolution is logged in step 1903. The calibration 
position (fixing point 1, 2 or 3) is recorded, 1904 and 1906. Two-dimensional 
images i = 1 to 1 = 10 are captured (1907, 1908, 1909). Image capture Is 
repeated for the remaining calibration fixing positions (1910, 1905). Following 

20 image capture each calibration position process is repeated for resolutions R + 1 
until R = n (1911). Having established a set of calibration values it Is possible, 
knowing the x, y and z offset of the light sources in a projection unit relative to an 
image plate, to conduct an image analysis by implementing a set of orientation 
values to calculate spatial and positional data from an image captured with a 

25 plurality of optical markers produced by light beams from light sources in a 
projection unit associated with an image plate. 



In this way calibration values for imaging devices, such as digital cameras, 
can be calculated. Each set of calibration values Is specific to one specific image 
30 plate when used in conjunction with a specific projection unit and mounting 
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means. Ideally a selection of calibration values for different digital cameras is 
provided in a software package such that the user can tailor that package to the 
specific hardware used to capture the image. Calibration values can also take 
account of different projection units having different spatial and angular light 
beam projection values. Referring to Fig. 15, once a set of calibration values 
have been established the next step in detennining three-dimensional positional 
data of regions of reflectivity in a field of view is to identify in the captured two- 
dimensional image the location of the at least one pixel within which the optical 
marker is located. Essentially, this is a search for laser points whicin can be 
carried out either manually wherein a two-dimensional image (e.g. BMP, JPEG, 
MPEG, GIF or other file type) is uploaded to a computer for viewing. Manual 
location of the points can be canied out by the user wherein the user identifies 
the optical markers within the. two-dimensional captured image. Alternatively, an 
automatic search for the optical markers can be conducted searching the image 
for a particular wavelength of light, typically of a wavelength of 635nm. A Fourier 
transform or colour based search are alternative search methods. Once the 
location of all the optical markers has been identified within the two-dimensional 
captured image the orientation step (step 3 in Fig. 15) can be conducted. 

Fig. 29 illustrates a process of searching for optical markers in a two- 
dimensional image of a field of view containing a plurality of optical markers in 
said field of view. A two-dimensranal image is loaded firom a pixel an^y 2901, a 
search of the image throughout it's height Is perfomied 2902 and a search 
throughout the image width is performed 2903 until a colour match, typically for 
an optical marker of wavelength 635nm is obtained 2904. Having found a colour 
match, data relating the x and y position of the matched pixel is stored 2905. The 
search is then progressed in a spiral search pattern 2906 until all of the optical 
markers have been located and the appropriate data stored. The process is then 
repeated for the next image 2907. 
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As the method of determining three-dimensional positional data of regions 
of reflectivity in a field of view is reliant on the relationship between the physical 
dimension of individual pixels in an image plate and the area of real world image 
in the field of view covered by that pixel it is necessary, in order to obtain the 
three-dimensional positional data required, to perform an orientation step. The 
orientation step determines the amount of relevant orientation values of the 
camera image plate center when a particular captured image was obtained. The 
orientation values comprise the tilt, pan, twist and height of the image plate from 
the regions of reflectivity in the field of view when the image was captured. 

The orientation values tilt, pan and twist have the following definitions with 
reference to Fig. 20 herein wherein 2001 represents a plane surface: 

• Twist is the deviation of the image plate fiioni an adjacent parallel 
plane as illustrated in Fig. 20A wherein C corresponds to the y axis 
entering the paper: 

• Tilt is the deviation of the image plate from a plane which is at right 
angles to the image plate and corresponds to Fig. 20B wherein C 
corresponds to the x axis entering the paper; 

• Pan is the deviation of the image plate from an opposite parallel 
plane wherein, in Fig. 20C, C corresponds to the z axis entering the 
paper. 

Fig. 21 is a representative illustration of the effect of pan and tilt and the real 
world space which is absorbed by a single pixel. Angle 2101 is the tilt and angle 
2102 is the pan of the image plate with respect to the plane of the field of view, it 
is readily noticeable that depending on the orientation of the image plate 2105 to 
the plane 2106 the area of plane absorbed by a single pixel (the pixels worth) 
differs depending on tiie orientation of Image plate to the plane 2106. 
Reciprocating this an^ngement for a fixed orientation of Image plate witii varying 
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tilt. pan and twist In the field of view, the area of real world space absorbed by a 
single pixel will vary with the orientation values which are therefore representative 
of a three-dimensional position of the regions of reflectivity in the field of view. 

The input data required to calculate the degree of tilt, pan and twist of the 
image plate center when the photograph (image capture) was taken is as follows: 

• Focal length of camera; 

• X and y pixel dimensions; 

• X and y pixel positions of the located optical markers in the image; 

• X and y image resolution; 

• Projection unit information e.g. angular relationship between beams 
and spatial relationships between beam sources, and relationship 
to image plate; 

In particular conditions the input data further comprises input from the user 
to discriminate between the field of view of a vertical plane or a horizontal plane. 
In the general condition, the operator/user does not have to discriminate between 
vertical or horizontal planes. 

The output data from the determination of orientation values is the: 

• Tilt deviation of the camera to the plane; 

• Pan deviation of the camera to the plane; 

• The height of the camera to the image plate center; 

• Twist deviation of the image plate; 

• Center axis of the plane at which the markers strike. 

Figs. 22 and 23A, B. C illustrate the effect of laser spots (optical markers) 
striking a ground plane in the field of view with a degree of pan and tilt applied 
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and the reflection back onto the image plate. Referring to Figs. 22 and 23A, B. C 
an example of the vector calculations to calculate the orientation of the image 
plate and projection unit are given belovy/. 



Orientation of Image Plate • Vector Calculations 
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2308 = S#P#_c 

2309 = S#P#_b 

Key to Fig. 23C 
5 2310 = P#CSep 

2311 = S#P#x_Hyp 

2312 = S#P#x_Adj 

2313 = s#p#e 

1 0 Determine camera and projection unit light source relationship 

P# C Sep = P# Sep + X Offset 

Determine Angles S# P# 9 for each point in each elevation 

(See Fig: 23A) 
15 For plan view Fig. 22 

S# P# 0 y = A Tan (S# P# y/f) 

For side view Fig. 22 

S#P#ex = ATan(S#P#x/f) 

20 

Determine associated length of light beams in plan elevation 

(See Fig. 23B) 

S#P#ly_C = S# Offset + 90 
S#P#ly_B = S#P#0-9O 
25 S#P#ly_A = 180(S#P#ly_C + S#Offset_B) 



Sin (S# P# ly_C) * (z Offset/Sin (S# P# ly_A)) 
Sin (S# P# ly_B) * (z OffseVSin (S# P# ly_A)) 
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Determine the center axis opposite and adjacent 

S# P# y_Opp = Sin (S# P# e y) * S# P#_c 
S#P#y_Adj= Cos(S#P#9y)*S#P#_c 

5 Determine associated length of light beams in side elevation 

(See Fig. 23C) 

S# P# x_Adj = P# C Sep/Tan (S# P# 0) 
S# P# x_Hyp = P# C Sep/Sin (S# P# 0) 

10 

Determine Pan angle 
Set1 

A = P1 CSep + P2CSep 
B = S1 P1_Adj + S1 P2_Adj 
15 Pan = ATanCVe) 



Set 2 

A = P1 C Sep + P2 C Sep 

B = S2 P1_Adj + S2 P2_Adj 

2 0 Pan = A Tan {\) 



Determine Tilt angle 

A = S2 P1 y_Opp - S1 P1 y_Opp 
B = S2 P1 y_Adj - S1 P1 y_Adj 
25 Tilt = ATan(%) 
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A = S2 P2 y_Opp - S1 P2 y_Opp 
B = S2 P2 y_Adj - S1 P2 y_Adj 
Tilt = ATanf/e) 

Determine iieigtit 

Center axis = Sin (S# C) * (S# P#_c/ Sin (tilt)) 
V = Center axis * Tan (tilt) 
Height = Cos (Pan) * V 

Pixel number 

Pi#Cy = ((Resy/2)-Pi#y) 
Pi#Gy = ATan(PjCy/f) 

Determine center axis distance in y only 

Pi # y_Adj = Height / tan (Pi # y__g) 
Pi#y_Hyp = ((Pi#y_Adj)'^2 + (Height)'^2)'^0.5 
Pi#y_c = Cos(Pi#y0)*Pi#y_Hyp 

Determine all x values using center axis value in y and pan 

Pi#Cx = ((Resx/2)-Pi#x) 
Pi#ex = ATan(Pi#Cx/f) 
Pl#x_g = 180-(Pi#xe + Pan) 
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For each Y Pi # y_c 
For each Pi # x_g 

Pi # X = Sin (IV Pan) * (Pi # y_c / Sin (Pi # x_g)) 
Next Pi # xjg 
Next Pi # y_c 

Determine all the remaining Y values using the previously calculated 
values including pan 

For each Pi#ye 
For PI # x 

Pi#x_adj = Cos(Pi#y0)*PI#x 

Pi # x_hyp - PI # x_aclj / Cos (Pi # y 9) 

Pi#y_adj = Pi # x_hyp * Cos (Pi # yjg) 

Determine height of each point 

Height Pi # x y = (Pi # x Hyp ^ 2 + Pi # y_Adj '^2) 0.5 
The above calculations are also suitable for matrix analysis. 

Fig. 24 illustrates a flow chart describing the Initial steps In detemiining a 
set of orientation values. Preferably, the steps of Figs. 24, 25. 26 and 27 are 
Implemented by the running of a suitable software package on a personal 
computer. Referring to Fig. 24, a captured image of a field of view containing 
therein a plurality of optical markers produced by a projection unit 103 in 
conjunction with an associated Image plate, e.g. a digital camera 102, is loaded 
In stage 2401 . The Image type Is recognised as one of a set of formats e.g. of 
either a BMP, JPEG or GIF image file. If the Image Is not in a recognised Image 
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format the message is invalid 2404. Once recognised as a valid image fomiat 
file, a.series of file data are assigned including file name, file header and details. 
Image type, width, height, width in bytes, image planes and bits per pixel. 

Refening to Fig. 25, once an image is loaded and recognised as described 
in Fig. 24 a check for the calibration values is made 2501, if the calibration values 
are present calibration data is read 2502 and the pixel positions are converted to 
millimeter measurements 2503, the resultant vectors of all the calibration points 
(optical markers) can be calculated 2504 and z and x world co-ordinates for each 
vector of all of the calibration points corresponding to the pan can be calculated 
2505. zx world details corresponding to the tilt 2506 and twist 2507 of the 
calibration points are also calculated. Calibration point pan, tilt and twist data 
thus detemiined Is stpred 2508. 

Refen-ing to Fig. 26, If the vector data relating to pan, tilt and twist is 
loaded 2601 it is read 2602. For a projection unit having at least one light source 
projecting six beams as illustrated in Fig. 1 two sets of optical markers are 
produced in the field of view. Refening to Fig. 2A set 1 comprises optical 
markers A, B and C and set 2 comprises optical markers D, E and F. Within 
each set a plurality of points (con-esponding to each optical marker) exists. A. 
determination of the adjacent difference in points in temns of an xy vector is 
calculated in step 2603 and adjusting for the separation between light sources 
2604 a set of values for pan deviation for set 1 and set 2 is determined (2605 and 
2606) and this data is stored 2607. 

Referring to Fig. 27 the calibration data is loaded 2701 and read 2702. 
The adjacent difference in point vectors yx and vectors yz is calculated 2703 and 
the opposite for each point in vector yz and vector yx is determined to establish a 
set of tilt values 2705 which are stored 2706. 
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The output of the orientation process described in Figs. 24 to 27 is the 
establishment of a set of orientation values describing the orientation of the 
image plate to the optical markers in the two-dimensional image of a field of view. 
Having established these orientation values it Is now possible to orientate the 
entire pixel array with respect to all regions of reflectivity in the field of view in the 
real worid with reference to the Image plate. This provides information about 
each pixel and the area in the real worid to which each pixel is associated (the 
pixel worth). Fig. 28 represents the real worid area that a pixel 2805 represents 
in the two-dimensional captured image. With reference to Fig. 28 the following 
statements hold tme: 

Where the height varies and the plane in the field of view retains a 
constant tilt, pan and twist: 

• As the image height grows in the z direction the area that a pixel 
represents in x and y Increases (compare 2801 and 2803); 

• As the image height reduces In the z direction the area that a pixel 
represents in x and y decreases. 

Where tilt varies and height, pan and twist remain constant: 

• As the image tilt angle grows the area that a pixel represents in x 
and y increases; 

• As the image tilt angle reduces the area that a pixel represents in x 
and y decreases. 

Where pan varies and height, tilt and twist remain constant: 

• If the pan angle is zero the distance in x and y remains constant for 
each y axis in x; 
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• As the pan angle is introduced the effect on the x and y distance 
varies; 

• If the pan angle is positive then the pixel worth to the left half 
increases and to the right decreases; 

• If the pan angle is negative then the pixel worth to the left half 
decreases and to the right increases. 

The orientation of the entire pixel an^ay in the real world with reference to 
the camera image plate con-esponds with Fig.15 step 4 (1510). 

Resultant stored data (2706, 2607, 2508) enables orientation of each pixel 
in the an-ay with respect to the real world. In order to calculate three-dimensional 
positional data of regions of reflectivity in the field of view it now remains to 
conduct an image analysis to obtain the world x, y and z positions for each pixel 
in the image plate. Image analysis comprises the determination of the 
convergence of points or planes in the space domain. In order to conduct the 
image analysis the following data is established: 

• X and y pixel dimensions; 

• x and z pixel position; 

• X and y image resolution; 

• Tilt, pan and twist deviation of the image plate to the plane (there is 
twist in the vertical plane); 

• Height of the camera (camera center axis) to the image plate 
center; 

• File name. 

Fig. 30 is a flow diagram illustrating the first steps in the image analysis. 
Calibrated pixel position and image data structure, including orientation values, is 
loaded 3001 . The set 1 pixel points are checked 3002. If they are in the same x 
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plane then determination of x and z with many y's is made 3005 and all vectors of 
X, y and z are then detemiined 3006, this data being stored 3007. WInere all set 
1 pixel points are not in the same x plane the calculation of x and y with many z's 
is made 3003 to calculate all vectors of x, y and z 3004 which is stored as data 
5 3007. 

Referring to Fig. 31 a checl< is made to see whether the resultant data has 
been loaded 3101, where the resultant has been loaded for each set and each 
point (3102 and 3103) the separation of a pixel from an optical marker is read 
10 3104 and an angle from the resultant data is also read. The adjacent is then 
calculated 3106 and stored as a data set comprising set number, point number 
and the conresponding adjacent 3107. This is repeated for all of the points on all 
ofthe sets (3108. 3109). 

15 Fig. 32 illustrates a flbw dilagram of the next steps of the image analysis. 

Loading of the resultant data is checked 3201, then for each set 3202 and each 
point 3203 separation distance from the optical marker data is read 3204 and a 
corresponding angle for a first point resultant data and a second point resultant 
data 3206 is read, a trigonometrical survey is perfomned (3207 - 3212) to 

20 establish a resultant value 3213 from which an adjacent, opposite and HYP Is 
established (3214 - 3216) which is stored as data set 3217. This is then repeated 
for all ofthe remaining points and sets 3218, 3219. 

Referring to Fig. 33 illustrating further steps in the image analysis a check 
25 for resultant data loaded 3301 is made. Where resultant data has been loaded 
analysis of each set and each point (3302, 3303) is made in respect of reading 
the separation of opposite and adjacent from the chosen set and point (3304) 
and the HYP calculated 3305. The set point and HYP stored as data 3306. This 
is repeated for all points and all sets (3307, 3308). 



30 
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The output data determined from the image analysis steps of Fig. 30 to 33 
are as follows: 

• A pixel array with corresponding world x positional data for each 
pixel in the captured image; 

• A pixel an-ay with all of the world y positional data for each pixel in 
the captured image; 

• A pixel anay with all the world z positional data for each pixel in the 
captured image. 

The real world x, y and z co-ordinates can then be established for each 
pixel corresponding to the three-dimensional positional data of each region of 
reflectivity in the field of view captured on the image plate. 

, . . This real world x, y and z stored data is the primary data upon which a • 
three-dimensional map/model of the captured image may be reconstnjcted. A 
further step in the image analysis process is to employ an edge detection module 
to systematically search the captured two-dimensional image for object and point 
edges in the captured image. The edge detection results can then be con-elated 
with the world x, y and z positional data to provide a finalised x, y and z data set. 
This finalised x, y and z data set can then be used to reconstmct three- 
dimensional models of the regions of reflectivity in the captured image, that is to 
reconstmct a virtual three-dimensional model of objects and surfaces in the field 
of view. 

The main steps in obtaining a three-dimensional positional data set of 
regions of reflectivity of a field of view starting with a captured two-dimensional 
image containing optical markers introduced to a field of view from a projection 
unit 103 where a set of calibration values have been detemriined are summarized 
in Fig. 34. Referring to Fig. 34, a search is made throughout the captured two- 
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dimensional image for each optical mariner 3401, a determination of the 
orientation values of tilt, pan, twist and height are made 3402. Each pixel is then 
evaluated in real world terms by establishing orientation values for each pixel 
3403. By correlation of a representation of the pixel area in the captured image 
with respect to the calculated orientation values the determination of height 
above a zero plane Is made 3404. This may be coupled with an edge detection 
perfomied throughout the captured two-dimensional image 3405. A known point 
(e.g. an optical marker) is then chosen 3406. From said known point the two- 
dimensional captured image is scanned to all points in a con^sponding x, y or z 
plane 3407 and each pixel in the image is correlated in the same chosen plane 
against said known point 3408. Step 3408 is repeated for all planes to assign x, 
y and z in the real worid to each pixel 3409. An en-or con-ection process is then 
earned out to connect for en-ors in x, y and z axis 3410. The assigned z values 
are then con-elated with the results of the edge detection to detemiine a finalized 
data set of said co-ordinates 341 1 which are then reintroduced to fonnulate a 
finalized three-dimensional positional data field set describing regions of 
reflectivity in the field of view of an image plate which has captured the two- 
dimensional image analyzed. This finalised data set can then be used to 
produce a virtual map/model In three-dimensions of the field of view. 

Fig. 35 illustrates a screen window of a software package for implementing 
the image analysis. The particular screen illustrated relates to the calibration 
values. Drop down menu 3501 provides the user with options to select one or 
more manufacturers of camera. The particular model can then be selected by 
selection of a serial number in drop down menu 3502. On the user selecting the 
correct manufacturer and model relating to the camera used to capture the image 
which is being analyzed, a set of previously determined calibration values is 
retrieved and displayed under the heading "Camera Data". A selection of the 
projection unit model is also made enabling retrieval of a predetermined set of 
calibration values for the projection unit which are displayed under the heading 
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"Laser Data" 3505. New camera and projection unit data can be added via input 
screens accessed through links 3506 and 3507. Once the user has chosen the 
correct data, the data entry is confirmed 3508. 

Whilst the invention has been described in terms of capturing an image 
with an Image plate comprising a plurality of pixels each capable of generating a 
pixel of data it is possible to primarily capture an image of a field of view with a 
plurality of optical markers 107 on a analogue type camera having a chemical 
based film format, said film format being of the type known In the prior art. A 
photographic Image thereby obtained on hard copy e.g. paper can be transferred 
by means of a digital scanner into a data set comprising a plurality of pixels 
fonning a two-dimensional image. By obtaining an equivalent set of calibration 
values relating the image plate to the projection unit used to obtain the image of 
the, field of view and optical mariners a set of orientation values can *be 
ascertained to enable an image analysis as previously described to obtain" a 
three-dimensional data set describing the three-dimensional positions of regions 
of reflectivity in the image captured. 

The present invention described above therefore enables a three- 
dimensional map of the regions of reflectivity in a field of view to be reconstructed 
from a two-dimensional image of that field of view captured on an image plate. 
Each region of reflectivity con-esponds to a point in the field of view from which 
light or other electromagnetic radiation is reflected. This reflected radiation can 
then be captured on the image plate. By obtaining positional data of these 
regions of reflectivity from a photographic image the ability to position accurately 
an object in the real world space from an image of that real worid space is 
provided. Non-exhaustive examples of the applications of the present invention 
include the following: 
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Surveying/mapping - an image of a building, or other area to be 
mapped, is captured on an image plate comprising a number of pixels 
where the Image includes optical markers projected finom a projection 
unit to enable rapid three-dimensional modeling and mapping. A 
single photograph can be taken and then either analysed on site by 
means of downloading to a laptop or sent back to head office via email 
for further analysis. Such analysis typically takes currently between 2 
to 4 minutes to produce a three-dimensional model and with increasing 
processor speeds an instantaneous or real time application for onsite 
mapping is possible. > 

Robotics - automated factory production lines commonly require 
positional data to locate one component with another. Where this is 
performed by automated robotic arrhs a rapid three-dimensional 
modeling is required to ensure proper location of components. 
Incorporating a camera into tiie robotic amis and an associated 
projection unit the required positional data can be detemnined in real 
time; 

Security - the invention has applications in tracking people and objects 
to monitor tiieir location which includes security applications and 
monitoring of stock; 

Marketing - tiie present invention also has applications in the 
recognising of specific objects e.g. people, and may be applied to 
monitor the number of people moving through a space or attending a 
retail outiet and automatically log their movements; 
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• Real time video Image modeling - the present invention is also 
compatible with image capture on a digital video camera wlierein a real 
time three-dimensional video image can be reconstructed. 

A further example of the applications of the present invention are 
illustrated by a further embodiment as follows. The projection of markers into a 
field of view is not limited to projection of visible light mariners but includes 
mariners produced by electromagnetic radiation beams extending throughout the 
electromagnetic spectrum from 10-^^ meters to 10"* meters. A particular example 
is the use of x-rays beams. Here, an x-ray source takes the place of the laser 
diode and one or more electromagnetic lens means is configured to focus each 
electromagnetic beam. One or more beam splitters may be included to produce 
a required number of.marl<e!:s,. Considering the applications In the medical field. 
Projecting a plurality of electromagnetic beams towards a subject, e.g. a human 
body, with a back plate of lead, enables x-rays to be reflected fi^m certain tissues 
through which the x-rays, cannot penetrate. Such tissues typically comprise 
bone. In this embodiment the range of wavelength of electromagnetic radiation 
being used to produce the markers projected Into the field of view must be 
compatible with the image plate such that the image plate is sensitive to that 
wavelength of electromagnetic radiation. By projecting a minimum of four x-ray 
markers, a model of the human body bone structure can be built up. By the use 
of appropriate radiation sensitive image plate arrays, focusing means and 
electromagnetic radiation sources, the principles of the present invention can be 
applied to electromagnetic radiation throughout the electromagnetic spectrum. 
Further potential uses of the present invention include thermal imaging using 
infi^-red radiation and long distance mapping by the use of radio waves. 

In a further embodiment of the present invention the inventors have 
understood the limits of utilizing a single imaging device. Any one imaging device 
has a maximum angle of view which determines the size of field of view. As a 
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result, large fields cannot be captured in a single image. To overcome this 
problem and to reconstruct detailed three-dimensional positional data of large 
fields of view it is necessary to take more than one image. This concept is 
illustrated in Figs. 36 and 37. 

5 

Referring to Fig. 36, a first image is captured at position 3601 and a 
second image is captured at position 3602. Each of the first and second images 
include a common reference point 3603. which may be extemally introduced, 
reconstruction of three-dimensional positional data is as described above. 

10 Having produced a three-dimensional data set the common reference point 3603 
enables the two reconstructed images to be aligned to provide a model of the 
combined field of view. This principle can be further expanded by capturing a 
plurality of images all having at least one comnron ref^rence point enabling data 
from those images to be coordinated to produob a- single data set for the 

1 5 combined field of view. 

Fig. 37 is an optical illustration of Fig. 36 illustrating first camera 3601 and 
second camera 3602 each capturing Images comprising at least one common 
reference point 3603. Data obtained from each captured image can be combined 
20 to produce a common three-dimensional positional data set to map regions of 
reflectivity in the combined field of view. 

Therefore by having a plurality of imaging devices, e.g. digital cameras or 
digital video cameras, each camera positioned to capture overiapping fields of 
25 view with at least one adjacent camera wherein common reference points can be 
determined in each overiapping field of view, an area larger than the field of view 
of a single camera can be mapped. By transmitting information between camera 
databases, movement of an object in a particular area may be anticipated. 
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Claims: 

1 . A method of obtaining multi-dimensional positional data from a field 
of^view, said field of view cx)ntaining a plurality of regions of reflectivity, said 
method comprising the steps of: 

projecting into said field of view a plurality of detectable markers, each 
marker produced by incidence of an electromagnetic beam on a region of 
reflectivity, 

wherein an angular relationship between said beams is known and a 
spatial relationship between the plurality of beams is also known at a source of 
said beams; and 

capturing a twoTdimensional image of said field of view and said mari<ers 
on an image plate, said image plate comprising an array of elements each 
capable of generating a pixel of data; and 

identifying said mariners in the captured image and using said known 
spatial and angular relationships to detenmine a set of orientation values 
describing the orientation of said image plate to regions of reflectivity in said field 
of view; and 

using said orientation values to reconstruct a set of multi-dimensional data 
describing the position of said regions of reflectivity. 

2. A method as claimed in claim 1, wherein multi-dimensional 
comprises uni-dimensional, two-dimensional or three-dimensional. 

3. A method as claimed in claim 1 or claim 2, wherein the spatial and 
angular relationship between said image plate and said sources is also known. 
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4. A method as claimed in any of claims 1 to 3, wherein said image 
plate comprises a CCD or CMOS array. 

5. A method as claimed in any of claims 1 to 4. wherein said 
orientation values comprise the pan of said image plate with respect to regions of 
reflectivity In said field of view, the pan being the deviation of the innage plate 
from an opposite parallel plane. 

6. A method as claimed in any of claims 1 to 5, wherein said 
orientation values comprise the twist of said image plate with respect to regions 
of reflectivity in said field of view, the twist being the deviation of the innage plate 
firom ah adjacent parallel plane. 

i\ A method as claimed in any of claims 1 to 6, wherein said 
oriehtatibh values comprise the tilt of said image plate with respect to regions of 
reflectivity in said field of view, the tilt being the deviation of the image plate from 
a plane at right angles to said image plate. 

8. A method as claimed in any previous claim, wherein said 
orientation values comprise at least one of the pan, twist, tilt and height of said 
image plate with respect to regions of reflectivity in said field of view. 

9. The method as claimed in any of claims 1 to 8, wherein said step of 
using said orientation values to reconstruct a set of multi-dimensional data 
describing the position of said regions of reflectivity comprises the steps of: 

using said orientation values to evaluate each pixel in the real worid by 
determining the degree of pan and/or twist and/or tilt for each pixel; 
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assoclating each said pixel of data with a said region of reflectivity and 
determining the real world position of said region of reflectivity. 

1 0. The method as claimed in daim 9, further comprising the step of: 
repeating said evaluation of each pixel for heights above a zero plane. 

1 1 . The method as claimed in claim 9 or 1 0, further comprising the step 
of selecting a pixel having known three-dimensional positional data and scanning 
to all three-dimensional planes to test that each pixel matches a set of three- 
dimensional co-ordinates. 

12. The method as claimed in any of dainis 9 to 11, wherein said step 
of using said orientation values to reconstruct a set of three-dimensional data 
describing the three-dimensional position of said regions of reflectivity further 
comprises the steps of: 

applying an edge detection function to the two-dimensional captured 
image; and 

checking the three-dimensional data against the edge detection results to 
establish a finalised set of three-dimensional positional data describing the three- 
dimensional position of said regions of reflectivity. 

13. The method as daimed in any previous claim, wherein said step of 
identifying said markers in the captured image comprises the step of: 

conducting a search of the captured two-dimensional image to identify 
pixels receiving reflected radiation of a particular wavelength. 
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14. The method as claimed in any of claims 1 to 13, further comprising 
the step of mapping the three-dimensional data obtained to reconstruct a virtual 
three-dimensional representation of the field of view. 

"? 15. The method as claimed in any of claims 1 to 14, wherein a said 

detectable marker captured in said two-dimensional image is configured to cover 
an area of between one and four pixels on said image plate. 

16. A method as claimed in any of claims 1 to 15, wherein said 
10 orientation values comprise one or more vector measurements of said regions of 

reflectivity. 

17. A computer prograni product directly loadable Into the intemal 
memory of a digital computer comprising software code portions for performing 

15 the steps of: v , ' 

identifying said markers in the captured image and using said known 
spatial and angular relationships to calculate a set of orientation values 
describing the orientation of said image plate to regions of reflectivity in said field 
20 of view; and 

using said orientation values to reconstruct a set of multi-dimensional data 
describing the position of said regions of reflectivity; 

2 5 said steps as claimed in claim 1 . 

18. A computer program product directly loadable into the intemal 
memory of a digital computer comprising software code portions for perfomiing 
the steps of any of claims 9 to 14. 

30 
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19. A method of determining the distance between regions of 
reflectivity in a field of view and an Image plate, said method comprising the steps 
of: 

projecting into said field of view at least one detectable marker, said 
marker produced by Incidence of an electromagnetic beam on a region of 
reflectivity: 

wherein angular and spatial relationships between the source of said 
beam and said image plate is known; and 

capturing a two-dimensional image of said field of view and said marker on 
an image plate, said image plate comprising an anray of elements each capable 
of generating a pixel of data; and 

identifying said marker in the captured image and using said known spatial 
and angular relationships to detennine a set of orientation values describing the 
orientation of said image plate to regions of reflectivity in said field of view; and 

using said orientation values to determine the distance between said 
image plate and one or more of said regions of reflectivity. 

20. Apparatus for the projection of a plurality of detectable markers 
onto a field of view, wherein said markers are capturable as part of a two- 
dimensional image of said field of view fomied on an image plate, said image 
plate comprising an array of elements each capable of generating a pixel of data, 
said apparatus comprising: 

a projection unit comprising at least one electromagnetic radiation source, 
said at least one source configured to produce a plurality of electromagnetic 
beams, each beam producing a said detectable marker on incidence with a 
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region of reflectivity in said field of view, wherein an angular relationship between 
the projected beams is known, a spatial relationship between the plurality of 
beams is also known at a source of said beams; and 

5 mounting means configured to engage said projection unit; 

said mounting means further comprising means to locate said image plate 
in a fixed spatial and angular orientation; 

10 wherein said projected beams are configured to project said mariners onto 

said field of view to provide information for the analysis of a captured two- 
dimensional image of said field of view and markers in order to establish a set of 
orientation values describing the orientation of said image plate to regioris of 
reflectivity In said field of view, said orientatton values enabling the reconstruction 

15 of multi-dimensional data describing the position of regions of reflectivity in said 
' field of view. 

21. Apparatus as claimed in daim 20, wherein multi-dimensional 
comprises uni-dimenslonal, two-dimensional or three-dimensional. 

20 

22. Apparatus as claimed in claim 20 or 21, wherein said image plate 
comprises a CCD or CMOS an-ay. 

23. Apparatus as claimed in any of claims 20 to 22, wherein said image 
25 plate is comprised by a digital camera or scanner. 



25. Apparatus as claimed in any of claims 21 to 24, wherein said 
plurality of beams comprises at least four beams. 
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25. Apparatus as claimed in any of claims 20 to 24, wherein said 
markers form a predefined pattem on said field of view. 

26. Apparatus as claimed in any of claims 20 to 25. wherein said 
5 electromagnetic radiation comprises radiation of a wavelength in the range 10"^' 

m to 10-^ m. 

27. Apparatus as claimed in any of claims 20 to 25. wherein said 
electromagnetic radiation comprises visible light of a wavelength of 635nm. 

10 

28. Apparatus as claimed in any of claims 20 to 27, wherein said image 
plate is comprised by a digital camera, said mounting means configured to 
engage said digital camera for location otsaid digital camera in fixed spatial and 
angular orientation to said projection unit. 

15 ■■ . ■ ■ 

29. A device for projecting a plurality of markers onto a field of view, 
said markers capturable on an image plate comprising an anay of elements each 
capable of generating a pixel of data, the captured two-dimensional image 
configured for the reconstruction of a set of multi-dimensional data describing the 

20 position of regions of reflectivity in said field of view from said captured two- 
dimensional image, said device comprising: 

at least one electromagnetic radiation source, said source configured to 
produce at least one electromagnetic radiation beam, said beam producing a 
25 said marker on incidence with a region of reflectivity in said field of view; and 



means to focus said beams; 
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wherein an angular relationship between said beams and a spatial 
relationship between said sources is known, the markers thereby fonning a 
predefined pattem In said field of view. 



radiation source comprises at least one laser. 

31. A device as claimed in claim 29 or 30, wherein said 
electromagnetic radiation source produces light of a wavelength in the range 10'^® 

10 mtolO'^m. 

32. A device as claimed in daim 29 or 30, wherein said 
electromagnetic radiation source produces visible light of a wavelength of 635nm. 

15 33. A device as claimed in any of claims 29 to 32. wherein said image 

plate fonrns part of a digital camels. 

34. A digital camera comprising a device as claimed in any of claims 29 
to 32 wherein said image plate is formed by the image plate housed within said 

20 digital camera. 

35. A method of detemnining a set of calibration values for an image 
plate comprising an array of elements each capable of generating a pixel of data, 
said image plate configured for use in capturing a two-dimensional Image of a 

25 field of view comprising a plurality of markers, said set of calibration values 
enabling the determination of multi-dimensional positional data describing the 
position of regions of reflectivity in said field of view, said method connprlsing the 



5 



30. A device as claimed in claim 29, wherein said electromagnetic. 



steps of: 
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locating an imaging device in a first fixing position, said imaging device 
housing said image plate; and 

capturing a two-dimensional image of a reference plane; 

wherein said image plate is substantially parallel to said reference plane. 

36. The method as claimed in claim 35, wherein following capture of an 
image at a first fixing position, said imaging device is relocated in at least one 
second fixing position from which at least one second image is captured. 

37. The method as claimed in claim 35 or 36, wherein said reference 
plane .cpiTiprises a planar surface divided into a grid having known dimensions. - 

38., The method as claimed in any of claims 35 to 37, wherein said, 
image plate comprises a CCD or CMOS aaay. 

39. The method as claimed in any of claims 35 to 38, further 
comprising the step of establishing a set of calibration values specific to said 
image plate. 

40. The method as claimed in claim 35, wherein said set of calibration 
values comprise one or more values taken from the set of: 

X, Y and Z ofi^sets of said image plate ftiom the fixing position; 
X, y pixel dimensions; 



focal length; 
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image plate width and height; 

X and y image plate view angles. 

5 41. Apparatus for determining a set of calibration values for an image 

plate comprising an aray of elements each capable of generating a pixel of data, 
said Image plate configured for use in capturing a two-dimensional image of a 
field of view comprising a plurality of markers, said set of calibration values 
enabling the detemiination of multi-dimensional positional data describing the 
1 0 position of regions of reflectivity in said field of view, said apparatus comprising: 

means to locate an imaging device in at least one fixing position, said 
imaging device housing said image plate; and 

15 a reference plane located in said field of view. 

42. Apparatus as claimed in claim 41, wherein said reference plane 
comprises a planar surface substantially parallel to said image plate and further 
comprising a grid of known dimensions. 

20 

43. Apparatus as claimed in any of claims 41 or 42, wherein said image 
plate comprises a CCD or CMOS array. 

44. The method of using a single camera device to obtain three- 
25 dimensional positional data from a two-dimensional captured image, said two- 
dimensional captured image including a plurality of non-intaisive optical markers 
having known characteristics, said optical markers being capturable in a two- 
dimensional image of the field of view. 
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